Evaluation of Structured Collaborative Tagging for Web Service Matchmaking

Turning Web services into Semantic Web Services (SWS) can be prohibitively expensive for large repositories. In such cases collecting community descriptions in forms of structured tags can be a more affordable approach to describe Web services. However, little is understood about how tagging impacts performance of retrieval. There are neither real structured tagging systems for Web services, nor real corpora of structured tags. To start addressing these issues, in our approach, we motivated taggers to tag services in a partially controlled environment. Specifically, taggers were given application requirements and asked to find and tag services that match the requirements. Tags collected in this way were used for Web service matchmaking and evaluated within the framework of the CrossEvaluation Track of the Third Semantic Service Selection 2009 contest. As part of the lessons learned, we explain relations between description schema (SWS, tags, flat document) and matchmaking heuristics, and performance of retrieval in different search scenarios. We also analyze reliability of tagging system performance as related to taggers’/searchers’ autonomy. Finally, we identify threats to results’ credibility stemming from partial control of the tags collection process. M. Gawinecki ( ) G. Cabri Department of Information Engineering, University of Modena and Reggio Emilia, Modena, Italy e-mail: [email protected]; [email protected] M. Paprzycki Warsaw School of Management, Warsaw, Poland Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland e-mail: [email protected] M. Ganzha Institute of Informatics, University of Gdańsk, Gdańsk, Poland Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland e-mail: [email protected] M.B. Blake et al. (eds.), Semantic Web Services, DOI 10.1007/978-3-642-28735-0 11, © Springer-Verlag Berlin Heidelberg 2012 173 174 M. Gawinecki et al.